iT邦幫忙

2025 iThome 鐵人賽

DAY 24
0
佛心分享-IT 人自學之術

學習 LLM系列 第 24

Day24 測試不同生成模型

  • 分享至 

  • xImage
  •  

在相同的檢索結果 + prompt 下
分別使用:

  • 一個 flan-t5-small
  • 一個 flan-t5-base

比較:

  • 回答語氣(口語 / 正式 / 結構)
  • 回答資訊的完整程度
    實作 :
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

def generate_answer(model_name, context, query):
    print(f"\n===== 測試模型:{model_name} =====")
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

    # 英文 prompt
    prompt = f"""
You are a helpful assistant. 
Here is the FAQ:

{context}

Question: {query}

Answer in English, short and clear:
"""

    inputs = tokenizer(prompt, return_tensors="pt", truncation=True)
    outputs = model.generate(**inputs, max_new_tokens=200)

    answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
    if answer.strip() == "":
        print("⚠️ 模型輸出為空")
    else:
        print("✅ 回答內容:", answer)
    return answer


# 測試資料
context = "If the product has defects, please take a photo and go to the customer service center to fill out a form. We will handle replacement or refund as soon as possible."
query = "What should I do if my product has defects?"

# === 測試 flan-t5-base ===
ans_base = generate_answer("google/flan-t5-base", context, query)

# === 測試 flan-t5-small ===
ans_small = generate_answer("google/flan-t5-small", context, query)

結果 :
===== 測試模型:google/flan-t5-base =====
✅ 回答內容: Take a photo and go to the customer service center to fill out a form.

===== 測試模型:google/flan-t5-small =====
✅ 回答內容: Take a photo and go to the customer service center to fill out a form. We will handle replacement or refund as soon as possible.

比較 :

  • flan-t5-base : Take a photo and go to the customer service center to fill out a form.
    • 內容正確:模型成功抓到了 FAQ 的核心資訊
    • 輸出長度較短:只回覆了處理流程的前半段
  • flan-t5-small : Take a photo and go to the customer service center to fill out a form. We will handle replacement or refund as soon as possible.
    • 內容完整:除了流程外,還補上了「我們會儘快處理退款或換貨」
    • 語氣更接近 FAQ 原文,而不是壓縮摘要

上一篇
Day23 改善檢索
下一篇
Day25 處理長文件
系列文
學習 LLM25
圖片
  熱門推薦
圖片
{{ item.channelVendor }} | {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言